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ABSTRACT 

Because universities in Canada, the U.S.A., and other Western countries are facing 
declining real income and increasing demands for greater accountability, they 
need to use resources more carefully and undertake planning activities more 
seriously. By obtaining information from reviews of academic and support units, 
universities are in a better position to meet these challenges. Most experts agree 
that reviews aimed at improvement of programs and activities are more successful 
when conducted under the authority of the university rather than an external 
agency. 

Attention needs to be paid to the establishment of an appropriate administra¬ 
tive mechanism that can maximize benefits relative to costs. This article describes 
the policies and procedures adopted in 1980 by The University of Alberta to 
allow for a continuous system of reviews of all academic and support units. 
Establishment of a President’s Advisory Committee with a minimum of formal 
procedures has led to a successful system under which 51 units are now in various 
stages of the review process. The procedures used at The University of Alberta 
may be worthy of scrutiny by other universities contemplating such activity. 


RESUME 

Au Canada, aux Etats-Unis et en d’autrespaysoccidentaux, les universites doivent 
faire face aux problemes de la diminution de leurs revenus reels, des exigences en 
matiere de comptes rendus plus satisfaisants de Tutilisation de leurs fonds et de 
la necessite de faire de plus grands efforts de planification. Pour etre plus en 
mesure de relever ces defis, les universites peuvent profiter des informations 
recueillies au cours des examens de I’etat actuel de leurs unites academiques et 
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de leurs services de soutien. La plupart des experts reconnaissant que de tels 
examens periodiques visant a. I’amelioration des programmes et des activites 
connaissent un plus grand succes quandc 'est liiniversite elle-meme qui les effectue 
plutdt qu ’un organisme externe quelconque. 

II faut avoir grand soin de mettre au point un mecanisme administrate propre 
a apporter le maximum d’avantages par rapport aux couts. Le present article 
decrit la politique et les procedures adoptees en 1980 par I’Universite de l’Alberta 
dans le but de faciliter un systeme permanent d’examen de I’etat actuel de toutes 
ses unites academiques et de tous ses services de soutien. La mise sur pied d’un 
Comite consultatif du Recteur ayant le moins possible de procedures formelles a 
donne naissance a un systeme d’examen qui produit de bom resultats et grace 
auquel 51 unites sont en vote d’etre soumis a I’examen ou I’ontdejd ete. Toute 
autre universite qui envisage une telle entreprise aurait interetaetudier soigneuse- 
ment le systeme en vigueur a I’Universite de VAlberta. 


SYSTEMATIC REVIEWS OF UNIVERSITY PROGRAMS AND UNITS 

Universities and colleges throughout North America have become increasingly 
involved during the 1970s and 1980s in reviews of their activities. This trend has 
resulted from (1) the need for more and better information on which to base 
budget and planning decisions, (2) the need to demonstrate to governments and 
the public that the institutions take accountability seriously, (3) the recognition 
that reviewing should be an integral part of the operation of an organization, and 
(4) the recognition that reviews can have important positive consequences. 
Although some universities and colleges outside North America, notably The 
University of London and The Australian National University, have carried out 
program reviews, most of the activity has occurred in the United States and 
Canada. The pertinent literature mainly reports procedures used in the United 
States, while those in Canada have received little formal academic attention. 
Further, most universities still tend to conduct reviews on an ad hoc basis in 
response to specific demands rather than on a systematic, on-going basis organized 
by a standing committee or administrative group. 

This article reviews selected literature on the topic of academic reviews and 
describes the review system at The University of Alberta. Beginning in 1980 all 
academic and support units are required to be reviewed under procedures organized 
by a President’s Advisory Committee on Campus Reviews. The Alberta approach, 
which was developed after lengthy deliberation, is seen by us to be a useful 
model that could be adapted by other Canadian universities and colleges. The 
awareness and possible adaptation of viable models can help to reduce the time 
that institutions contemplating reviews spend in developing their procedures. 
Without appearing to be presumptuous, we think that the following recommen¬ 
dation from Craven (1980) is appropriate: 
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A number of institutions currently have exemplary approaches to 
academic program evaluation. The concepts, guidelines, and operating 
experience that accompany these evaluation approaches should be 
shared more widely with other institutions, (p. 452) 

LITERATURE REVIEW 

A substantial amount of literature on evaluation is available. Although much 
relates to elementary and secondary education, a growing number of books and 
articles focus on the postsecondary sector. For example. Craven (1980) has pro¬ 
vided an excellent article dealing with the concepts and issues involved in program 
review as well as different approaches which can be undertaken. However, little 
has been published on evaluation and reviews in Canadian universities and colleges. 
One exception is an article by Bdlanger and Tremblay (1982): they discussed 
the role of committees at the Universite de Montreal in examining both academic 
and administrative units and making recommendations for reductions in course 
offerings and teaching personnel in response to diminished provincial funding. 
Another Canadian experience is reported in The Shrinking Maze which outlines 
the review procedures used in 1979-80 by The University of Calgary (1980). 

Definitions 

Certain terms should be defined early in this discussion. Evaluation is assumed to 
be an assessment of the extent to which a performance meets preselected criteria. 
Scriven’s distinction between summative evaluation (conclusion-oriented) and 
formative evaluation (development-oriented) is well known. A review is usually 
meant to be much more comprehensive than an evaluation, since it encompasses 
a detailed examination of historical and current aspects as well as of proposals 
for future development. Program is taken to refer to either a particular academic 
program, such as the B.A. program, or more commonly to a budgetary unit such 
as a department or group of departments, as proposed by Gilmour (1980): 

To my way of thinking, academic departments ultimately are a better 
unit of analysis than degree programs, because they are the activities 
to which institutional resources are allocated and form the basic 
building block of the operations and management structure at most 
higher education institutions, (p. 1) 

Probably the use of program review by Arns and Poland (1980) would be 
widely accepted, that is, “a searching comprehensive evaluation of an existing, 
coherent set of academic activities” (p. 269) involving both a self-study phase and 
a formal review phase conducted by a team of experts that may include outsiders. 

Purposes 

The following list of purposes behind the conducting of program reviews has 
been compiled from several different sources: 

1. To inform departmental and institutional decision makers about a program’s 
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strengths and weaknesses, as the first step in a process designed to improve its 
quality (Council of Graduate Schools in the United States, 1979, p. 2 ; Peterson, 
1980, p. 154; Barak and Berdahl, 1978, p. 56; Dressel and Cammack, 1972, 

p. 288). 

2. To establish a program’s status relative to specified standards or in relation to 
the performance of other similar programs (Council of Graduate Schools in 
the United States, 1979, p. 2). 

3. To assist the planning process (Peterson, 1980, p. 124). 

4. To arrive at judgments about a range of decision alternatives concerning the 
installation, continuation, modification, or termination of that program (Jany 
and Keller, 1981, p. 3; Peterson, 1980,p. 154;Barak and Berdahl, 1978,p. 56). 

5. To assist an institution when it is forced to reduce expenditures and/or increase 
efficiency (Barak and Berdahl, 1978, p. 56). 

6. To demonstrate accountability and responsiveness (Barak and Berdahl, 1978, 
P- 56). 

7. To protect students (Barak and Berdahl, 1978, p. 56). 

Several writers have noted that internal and external reviewers usually have 
different purposes: For example, Jany and Keller compared different emphases 
by internal and external reviewers: 

Institutional efforts in program evaluation have generally focused on 
learning or knowledge outcomes while external efforts have empha¬ 
sized fiscal austerity, management malpractice, consumer protection, 
and accountability to government agencies and society, (p. 5) 

Their opinion was supported by Folger (1980): 

The major faculty motivation for program review is to document the 
need for additional resources, which is in opposition to a state moti¬ 
vation to use program review as a basis for using existing resources 
more effectively, (p. 54) 


Procedures 

Craven (1980, pp. 444-449) has provided one of the most comprehensive listings 

of matters that need to be considered when program evaluations are to be con¬ 
ducted. He classifies these matters under five general headings: 

1. Defining issues, participants, purposes and processes, for example, what 
information is needed and by what processes are judgments made? 

2. Designing information analysis and data collection, for example, what are the 
sources of the data and how will the data be analyzed? 

3. Deciding on interpretation and reporting, for example, what are the criteria 
to be used for evaluation of the program, and who receives the reports? 

4. Planning an audit, that is, how are the program evaluations themselves to be 
evaluated? 
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5. Implementing the design of the program evaluation. 

The effectiveness of program evaluation can be assessed, according to Craven 
(1980, p. 434), by these criteria: Are the “evaluation results useful in arriving at 
judgments about decision alternatives for a given purpose”? Is the quality of these 
decisions better because information from the reviews was considered? Are the 
selected decision alternatives actually implemented? 

Dressel and Cammack (1972, pp. 282-283) have paid particular attention to 
selection of the review committees. Are the members to be elected or appointed 
or a combination of both? What criteria should be used in their selection? Should 
peer respect and convictions be considered? They also mention many of the 
matters raised by Craven, but in addition emphasize wise publicity and clarity 
of role expectations for members of the review committees. 

Another important aspect was raised by Dressel and Cammack (1972): “Because 
self-study [review] usually avoids pinpointing individual weaknesses, it, there¬ 
fore, is not likely to correct problems created by poor leadership” and that a 
review is “no substitute for lack of administrative willingness to deal forthrightly 
with a well-defined issue” (pp. 277-278). That is, reviews should not be used to 
solve problems that should be handled by routine administrative procedures. 

Trends 

The extent of use of program reviews in U.S. colleges and universities was surveyed 
by the National Center for Higher Education Management Systems (NCHEMS) 
in 1980. Barak (1982) noted that the Carnegie Council’s 1975 speculation that 
“institutional administrators would be relying much more heavily on program 
and course review in the future” has been borne out by data obtained in the 
NCHEMS survey (pp. 34-35). Of 1,082 institutions surveyed, 882 (82 percent) 
stated that they had some formal program-review process, with 43% of these 
having adopted their present policies and procedures since 1975. Barak also 
observed that “large research universities are most apt to conduct internal reviews” 
and that “universities that rank high in the various peer ratings are likely to show 
a similarly high usage of internal program reviews” (p. 35). McCorkle and Archi¬ 
bald (1982) concluded that “program review is now more formal, more frequent, 
and more comprehensive in scope and depth than in the past,” that the criteria 
for judging programs have changed, and that “the procedures and personnel 
involved have grown in number and complexity” (p. 143). 

Similarly Miller (1979) stated that evaluation will continue to increase in 
higher education during the 1980s and cautioned that “if public postsecondary 
institutions do not undertake rigorous evaluation, likely it will be done for them” 
(pp. 1-3). The continuing and increasing need for reviews was also emphasized 
by Arns and Poland (1980); they built their case upon the requirement that 
universities, which are complex organizations of “considerable inertia,” must 
evaluate to decide how to change in response to societal changes (p. 268). 

Another important aspect emphasized by Shirley and Volkwein (1978), is 
that, though the need to set priorities has always existed in universities, it was 
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not recognized until “‘steady-state’ financing” appeared (p. 473). “Hastily con¬ 
stituted committees” then were faced with the setting of priorities. Arns and 
Poland (1980) also observed that during the 1960s universities faced few program 
choices because new programs received the necessary resources and constraints 
were rare (p. 273). With changed circumstances, the need to incorporate some 
ongoing evaluation mechanism seems obvious. In the NCHEMS review, Barak 
(1982) noted that most institutions initiated reviews to improve their programs 
because knowledge of strengths and weaknesses was necessary before improve¬ 
ments could be planned (p. 39). Redistribution of resources and reduction of 
programs was the second most commonly cited reason, especially for contracting 
institutions. Barak also reported that “a number of colleges and universities are 
actively implementing planning processes that use program review to set priorities 
among institutional programs” (p. 38). 

Benefits and Costs 

What are some of the specific benefits from reviews? Hoyt (1980) suggested the 
following: they force examination of frequently ignored matters, provide quali¬ 
tative balance to normal quantitative indices, provide healthy involvement of 
program participants, point to constructive changes in response to negative find¬ 
ings, and create a positive institutional image (pp. 3-4). Dressel and Cammack 
(1972) considered that the strength of reviews done by an institution “lies in the 
involvement of those affected by the subject under study; gradual molding of 
receptivity for the changes proposed is a usual result of the study” (p. 274). 
Some academics have found the self-study phase to be at least as beneficial as 
the formal review and the review-phase activities at least as important as the 
reports. This relevant assessment has been made by Russo, Brown, and Rothweiler 
(1977): 

The act of asking questions of graduates, drop-outs, current students, 
and faculty about goals, objectives, educational process, and their 
relationship to each other causes each to pause to contemplate these 
matters. Specific program strengths and weaknesses are often identi¬ 
fied. Curricular revisions have been initiated or expedited as a result 
of the review process. Student information systems have been found 
seriously lacking and are being modified. The follow-up of graduates 
has been encouraged and is taking place in some departments for the 
first time. Questions and recommendations about academic quality 
and student flow have produced positive program revisions, (pp. 
297-298) 

Nevertheless, the value of the formal review phase should not be minimized. 
Arns and Poland (1980) stated that reviews contribute to the information, under¬ 
standing, mutual trust, and sense of stability needed to make wise decisions, and 
that “The most important outcome may well be the strengthening of confidence 
in decisions, their evolution, and their implementation” (p. 283). Another benefit 
was identified by Russo et al. (1977), who expressed the opinion that review 
information allows selective program termination rather than across-the-board 
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cuts that may threaten “the qualitative survival of our academic programs” 
(p. 292). This conformed with their view that “Future program decisions must 
be based on an integration of financial and academic data” (p. 292). 

McCorkle and Archibald (1982) have strongly argued in favor of routine 
reviews: occasional ad hoc reviews intensify stress, whereas “continued self-study, 
encouraged by senior administrators. . . can uncover opportunities for improved 
service in a positive context” (p. 148). Similarly, Craven (1980) urged that “as a 
matter of good academic practice ... a self-study of each academic program be 
conducted every five or six years” (p. 443). 

As to cost, certain administrative expenses in operating the review process are 
obvious — for example, salaries, honoraria, travel, subsistence, and telephone. 
Dressel and Cammack (1972) cautioned that much of the cost of reviews is 
“hidden or absorbed by unpaid overtime” (p. 274). But other costs and negative 
consequences may also be involved. Hoyt (1980) identified these to be (1) the 
poor planning and decision-making based on erroneous evaluations, which can 
weaken the institution and unjustifiably disrupt employees’ lives;(2) the paranoia 
resulting from adverse effects on morale; and (3) the replacement of intrinsic by 
extrinsic factors as motivators of behavior. 

To increase the benefits and reduce the costs, Hoyt advocated these approaches 
(pp. 4-6): 

1. Because the investment in time, effort, and money must be reasonable, the 
university should concentrate upon major objectives and collect only the 
most relevant data. 

2. The review system must be credible and acceptable to each segment of the 
institutional community. 

3. The validity of the program evaluation must be clear. 

4. Effective communication, especially with respect to early preparation of the 
report and avoidance of premature announcements, is essential. 

This list is expanded by Dressel and Cammack (1972) who felt that the impact 
of program review is affected by the state of readiness for the study, the clarity 
of the charge provided to the committee, the tact and leadership of the committee 
chairman, the availability and quality of staff assistance, cooperation in data 
collection, effective use of consultants, and the quality of reporting (p. 288). 
They also emphasize that continuing communication among all affected groups 
during the review is critical (p. 289). Concerning appraisal of the effectiveness of 
reviews, Dressel and Cammack cautioned that this appraisal should not be based 
upon how many recommendations are adopted, but upon changes made as a 
result of the overall review activity which may generate ferment leading to adop¬ 
tion of modified recommendations (p. 289). 

Some Specific Practices 

The best known North American experience with program reviews is that of 
The University of Illinois at Urbana-Champaign (1980). Its procedures have 
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influenced many other universities, especially through the involvement of its 
staff members in consulting and in conducting conferences. The review processes 
at The University of Vermont and Ohio State University have also been well 
reported. Arns and Poland (1980) noted that for these two universities “we made 
it clear that our purpose was program improvement, nothing more and nothing 
less” (p. 269). The Vermont and Ohio processes incorporated these major aspects: 
flexibility in collecting information, a self-study report, involvement of all depart¬ 
ment members, open communication, feedback, use of a coordinating committee 
composed of senior faculty from outside the department being reviewed, and 
development of a Memorandum of Understanding. DiBiasio and Ecker (1982) 
described this Memorandum, which concludes the review, as containing a 
mutually accepted statement of program objectives, steps to achieve these objec¬ 
tives, the time frame, and estimated costs (p. 15). 

The University of Saskatchewan took a different approach (Holmlund, 1980). 
In 1976 it began reviews of colleges (faculties) when the term of a dean had 
expired. The Board of Governors started the reviews to provide both it and senior 
administrators with information about relations, accomplishments, adequacy of 
resources, opportunities, and problems. A Supervisory Committee on College 
Reviews controlled the reviews, which were organized by the University Studies 
Group (equivalent to an office of institutional research). 

At The University of Calgary, which had earlier conducted some ad hoc reviews 
of faculties, the President in 1979 appointed a University Program Review 
Committee “to analyze the current status of the University and to develop an 
appropriate program for the decade ahead” (University of Calgary, 1980, p. i). 
This Committee consisted of the President (as Chairman) and eight academics of 
diverse rank from different faculties. Each faculty was studied in depth in 1979-80 
by the Committee members who were each assigned to two faculties. A draft 
analysis was prepared of organization, role, programs, opportunities and problems. 
Faculties were then requested to respond to this draft. 

A different practice occurs at Queen’s University where annual reviews of the 
Faculty of Engineering have been conducted for the last 25 years. Teams of three 
people — from industry, government and another university — spend two days 
each year reviewing departments in the Faculty as well as five departments which 
service Engineering, namely. Biology, Chemistry, Geology, Mathematics and 
Physics. The reviewing of geology departments was reported by Neale and Arm¬ 
strong (1980) to be a common practice in Canada, but only Queen’s, Memorial 
and Toronto used this practice on a yearly or biennial basis. Neale and Armstrong 
criticized the ad hoc nature of these reviews and lack of follow-up visits, and 
drew these conclusions from comments made at Queen’s, Memorial and Toronto: 

Administrators at these universities state that the regular visits of 
such committees and their frank unvarnished appraisals give them 
confidence in the operation of their geology departments. Depart¬ 
ment heads state that these committees introduce fresh new ideas 
into their programs, act as safety valves which defuse potentially 
explosive situations, and generally carry a great deal of weight in 
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their constructive recommendations to senior administration (p. 24). 

In some cases more than one campus is involved. For example, in 1975 the 
Board of Regents of The University of Wisconsin System delegated responsibility 
to its President for “formal ongoing program planning and review processes on 
its thirteen campuses” (Jany and Keller, 1982, p. 7). Other variations are also 
noted. Some universities use both internal and external reviewers on the same 
team, whereas others, for example. The University of Minnesota (Zimmerman, 
1981), use separate internal and external review committees. At Minnesota, where 
graduate reviews started in 1972, the internal committee produces a set of 
questions that the external committee can ask so as to make more efficient use 
of its time. 

Based upon experiences gained at these universities and elsewhere, several 
writers have identified aspects needed for successful implementation of program 
evaluation. Munitz and Wright (1980) named selective rather than comprehensive 
evaluation of programs, trust and genuine participation, a clear definition of 
evaluation objectives, and an overall framework of institutional planning priorities 
within which administrators can interpret the review outcomes (pp. 41-42). 

Opinions differ as to the extent to which the self-study report should concen¬ 
trate upon data and details. Arns and Poland (1980) recommended that interpre¬ 
tation of the data should be emphasized rather than the data themselves (p. 279). 
Most experts, however, agree that both objective data and qualitative assessment 
are required (for example, Miller, 1979; Gentile, 1980). 

Based upon the 1980 NCHEMS study cited, Barak (1982),pp. 36-47, extracted 
these generalizations: 

1. A few institutions include reviews of research programs and support depart¬ 
ments. 

2. Reviews are becoming more rigorous, a trend leading to improved results. 

3. More data are being incorporated into the review process. 

4. Reviews are becoming more important in institutional decision-making and 
planning. 

5. Outside peer faculty are usually involved in the review teams, spending one 
to five days on campus. 

6. Deans of faculties usually organize the reviews. 

7. Students, graduates, and former students usually serve as participants in 
surveys rather than as members of organizing or reviewing committees. 

8. The support of the president and trustees for the review process is essential. 

9. Programs are selected for review at convenient times (such as on retirement 
of a dean) or when their turn comes up in a cycle. Some institutions favor 
combining reviews with accreditation visits, whereas others discourage or 
forbid this practice. 

10. The concept of “quality” and its assessment in reviews leads to considerable 
disagreement. 
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11. Most reviews are perceived to be beneficial. 

In contrast to the well-established procedures used for academic program 
review at many universities, reviews of administrative/support units are in their 
infancy. The University of California provides a notable exception to this con¬ 
clusion (Griffin and Burks, 1976). At the Twin Cities Campus of The University 
of Minnesota, a Task Force for Research and Planning Information, set up by 
the Vice-President for Student Affairs, recommended adoption of the University 
of California’s structured review team procedure for support units which would 
have these key elements: 

1. Administration — assessment steering committee of four to six campus officers; 

2. Review teams — three to seven persons having a mixture of managerial skill, 
technical expertise, and campus knowledge; 

3. Staff — 0.5 to 1.5 FTE would administer the review and organize data collection; 

4. Scope — a rolling schedule of reviews of functional areas or departments; 

5. Procedures — reviewing data and conducting interviews was expected to take 
12-15 weeks; 

6. Report — clear recommendations would be specified; and 

7. Actions — line managers would react and develop implementation plans. 

This recommended procedure has not yet been implemented mainly because 

of “a severe budget crisis” (Matross and Delmont, 1982, pp. 17-21). However, 
The University of Minnesota (Twin Cities Campus) did review four other support 
units in the early 1980s using self-assessment and internal and external committee 
reports based on procedures developed by its Graduate School (Pflaum, 1982). 
Further evidence of interest in reviews of support/administrative units is provided 
by the 1981 decision of The University of Illinois at Urbana-Champaign to under¬ 
take comprehensive reviews of these units. Each Vice-Chancellor is to design an 
evaluation approach appropriate to his own units, as the procedure used for 
academic reviews was deemed to be unsuitable for the support/administrative 
area. Four types of reviews — desk, focused, strategic and comprehensive — can 
be used (McManus, Todd and Wilson, 1982). 

Classification of Models for Program Review 

A scheme for classifying program-review models has been proposed by Holdaway 
(1980). These range from the participating governance model, in which the senior 
elected decision-making body on campus selects the program-review committee, 
to the state model, in which the state/provincial education department does the 
selection. In between lie the executive model and the governing-board model, in 
which the president or board selects the committees and oversees the process. 
Universities must decide how the organizing committee for reviews is to be con¬ 
stituted and how this committee will relate to the committees actually conducting 
the reviews. One extreme position is exemplified by The University of Colorado 
at Boulder where in 1977 the fifteen members of the University Review Com¬ 
mittee separated themselves into five teams to review all campus units in one year. 
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At the other extreme, (the more usual approach) the organizing and review 
committees are clearly separated and have no common members. 

PROCEDURES AT THE UNIVERSITY OF ALBERTA 


Background 

The University of Alberta was founded in 1908 in Edmonton. It operated a 
campus in Calgary from 1951 until 1966, when The University of Calgary became 
an autonomous institution. Current enrolment at The University of Alberta is 
about 20,500 full-time undergraduate students enrolled in 16 faculties, 2,500 
full-time graduate students, and 4,000 part-time students. Figure 1 shows the 
relationships among its senior administrators, the Board of Governors, the General 
Faculties Council (GFC), and senior GFC planning committees. The GFC of 136 
members consists of ex-officio, administrators, other staff members, and students. 
Under The Universities Act, GFC has overall responsibility for most academic 
matters, although the Board of Governors has ultimate authority. 

In 1977 The University of Alberta, being concerned about the need to ensure 
that its allocation of resources to campus units was appropriate, established a 
University Priorities Committee (UPC). The UPC concluded that planning during 
the rapid growth of the 1960s and 1970s concentrated on incorporating new 
programs and that “We had neither a system of priorities nor adequate data on 
which priorities could be established.” One UPC recommendation was that “The 
Board of Governors and General Faculties Council jointly approve in principle a 
system of reviews of academic programs, academic units, and administrative and 
service units.” After extensive discussion, the university endorsed two initiatives: 
(1) the Office of Institutional Research and Planning was to develop an improved 
data base to assist decision making, and (2) a system of reviews of academic 
programs and service units was to be implemented. 

A Joint Committee of the Academic Development Committee and Deans’ 
Council was established to develop terms of reference for the system of reviews. 
This Joint Committee, which consisted of the Vice-President (Academic) as chair¬ 
man, two deans, and four professors, met for several long sessions during which 
the practices of other universities were evaluated, especially those of The Univer¬ 
sity of Illinois. It recommended procedures that would be as simple as possible 
and would not interfere with the university’s established decision-making structure. 
Its final recommendations, which were approved with only one change by GFC 
and the Board of Governors, were these: 

1. The goal of the reviews should be production of information that serves as a 
reliable basis for planning and improvement of programs. 

2. The administrative structure should be readily understood, effective, efficient, 
responsive, flexible, credible, and realistic. 

3. A President’s Advisory Committee on Academic Reviews (PACAR) should 
be established, rather than a committee of General Faculties Council. 

4. PACAR should consist of a chairman, the Vice-President (Academic), and 
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Figure 1. Relationships among senior administrators and committees at The University of Alberta. 


two other members selected by the President because of their experience, 
judgment, and reputation. 

5. The primary tasks of PACAR should be as follows: 

a. to advise the President on the order in which units should be reviewed 
and on the composition of reviewing teams; 

b. to advise reviewing teams on the requirements of a particular review; 

c. to transmit reports of reviews to the President; and 

d. to keep the university community informed about its work. 

6. Individual Unit Review Committees should be appointed, consisting of one 
member from outside the University but of the same discipline, one from a 
related discipline within the University, and one from the University at large. 
(The number of “outsiders” was later changed to two by the General Facul¬ 
ties Council and the Board of Governors.) 

7. Each unit to be reviewed should prepare a self-study report as a starting 
point for the review process. 

8. The reviews should be conducted over a period of three to five years, with 
the order of selection of units being determined by various considerations 
such as change of dean or chairman. 

9. Each Unit Review Committee should be expected to consult actively with 
undergraduate and graduate students, academic and non-academic staff, and 
recent graduates and employers of graduates where appropriate. 
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10. Various sets of recommendations should be sent to different administrators 
and committees on campus for action or discussion. 

11. Responsibility for monitoring action taken with respect to recommendations 
of the Unit Review Committees should rest with the Vice-President (Academic). 

In response to these approved recommendations, the President in 1980 
appointed a PACAR consisting of (a) full-time Chairman — a former Chairman 
of the Department of Chemistry who had recently retired, (b) the current Dean 
of the Faculty of Graduate Studies and Research, (c) a former Dean of Engineering, 
who was still on staff, and (d) the Vice-President (Academic). A deliberate deci¬ 
sion was made to select a chairman who was not a member of the university’s 
central administration. The Director of Institutional Research and Planning also 
attended the planning meetings of PACAR. 

Concurrent discussions were held by the President and Vice-Presidents con¬ 
cerning how the administrative and service units should be reviewed. Three possi¬ 
bilities were discussed: 

1. A separate review committee could be established. 

2. A subcommittee of PACAR could be established to which appropriate people 
could be seconded temporarily. 

3. PACAR could assume this extra responsibility. 

The third possibility was adopted. The Director of Technical Services was added 
to PACAR, and the name modified to the President’s Advisory Committee on 
Campus Reviews (PACCR). The benefits of this unified approach of having all 
reviews coordinated by PACCR were that (1) a global view could be obtained of 
all interunit relations and (2) administrative costs would be minimized. However, 
this added responsibility increased the workload of PACCR members. During 
1981-82 the Deans of Engineering and of Graduate Studies and Research were 
succeeded by two other professors. Now that the review procedures have been 
well established, the Vice-President (Academic) attends meetings only when his 
presence is requested. 

Operating Principles 

Several important principles were endorsed by PACCR and the President and 
publicized during the early phases of organization. These principles were based 
in part upon opinions expressed in the literature cited above and the experiences 
of other universities. 

1. The University’s attitude in conducting reviews was to be one of “forward- 
looking change and the encouragement of high-quality scholarship” (University 
of Alberta, 1980b), that is, the reviews were to be positive exercises and not 
witch-hunts. 

2. Multiple benefits were to be expected, such as improved planning and aware¬ 
ness of operations. 

3. Considerable flexibility in the different phases was to be emphasized, with a 
minimum of directions being provided for the self-study and review committees. 
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Figure 2. Simplified representation of the overall review process at The University of Alberta. 


4. While the procedures were being developed, a limited number of units were to 
be reviewed. 

5. The PACCR and accreditation reviews should coincide whenever possible to 
avoid duplication. 

6. Sensationalism was to be avoided and confidentiality respected, especially as 
to comments submitted to the members of the Unit Review Committee and 
the responses of a unit and others to preliminary and final reports. 

The importance of the use of numerical data in the review process should be 
recognized but not overemphasized. For a self-study report to be useful, it must 
contain data relevant to such aspects as staff workload, student enrolments and 
registrations, budget, research, publications, and graduate activity. Such data need 
to be coupled with qualitative descriptions of activities before evaluation can take 
place. Consequently, a balance of data, descriptive information, and evaluation 
is necessary if reviews are to be fully effective. A requirement to provide detailed 
information in response to specific questions means that there is likely to be less 
time for serious reflection. 

The Overall Process 

Many aspects of the overall review process at The University of Alberta are 
reported in the terms of reference listed earlier. The specific approach taken is 
illustrated in Figure 2. When the self-study report is complete, it is sent to PACCR 
and the President. A recommendation for members of a Unit Review Committee 
is sent to the President by PACCR after consultation with the unit. Copies of the 
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report are sent to members of the Unit Review Committee. Typically, about a 
month later the committee meets on campus for an on-site visit of two, three, or 
four days. A series of interviews is arranged, after which the members write a 
report, which is expected within one to two months after the campus visit. This 
review report is sent to PACCR, and with the clearance of the President, the unit 
is invited to submit a response. The final official step occurs when the self-study 
report, the Unit Review Committee report, and the unit’s response are discussed 
by the Planning and Priorities Committee. Discussions between the President and 
the head of the unit can occur at any phase of the review process. 

Action can be taken by the President, usually in consultation with the Vice- 
Presidents and other appropriate administrators, at any stage of the process. 
That is, the President need not wait until the Planning and Priorities Committee 
has discussed all the reports, but can take appropriate action at any time after 
receiving the self-study report. The unit can make changes and take action before 
completion of the self-study phase, and possibly long after formal completion. 

PACCR provides advice sheets to units to be reviewed concerning the self- 
study and its report, the Unit Review Committee, and the on-site review. It also 
transmits standardized information provided by the Office of Institutional 
Research and Planning showing academic and other data for the previous ten 
years and budget data for five years. Advice sheets are also provided to students 
concerning how they can be effective in providing input to a Unit Review Com¬ 
mittee. Further, invitations to provide confidential written input are printed in 
Folio, a weekly publication of The University of Alberta. 

The self-study phase. The self-study report of the unit should include these 
aspects: history, current status, impending changes, future prospects, strengths, 
weaknesses, and limitations. Because program improvement is the preeminent 
objective, global rather than specific instructions are forwarded from PACCR, 
Examples of self-study reports from other departments are provided in confidence. 
To obtain the maximum benefit, the unit is asked to be open both to itself and 
the review team. 

The self-study phase has usually taken about six months. It involves a consi¬ 
derable amount of staff time, especially from the chairman. (This has been a 
common experience at other universities, as noted by Seagren and Bean in 1981.) 
The reports have usually been realistic and balanced with respect to statements 
of strengths, concerns, and problems. Our experience is that more comprehensive 
and revealing reports make the work of the reviewers more effective. 

Unit Review Committees. The Unit Review Committees consist of four people. 
Two are from the discipline (or similar support units) outside the university, one 
is an internal member from a related department, and one is from the university- 
at-large. This balance has proven excellent. The unit prepares a slate of about six 
possible external members and submits this list with resumes to PACCR. For 
each nominated member of a team there must be evidence of a high level of 
excellence in a chosen field and evidence of objectivity and trusted judgment. 
The unit is also invited to nominate candidates for the member from a related 
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department. One important principle is that all reviewers must be acceptable to 
the unit as well as to PACCR and the President. Each reviewer is supplied ahead 
of time with several items, including the self-study report. In 1981 the instruc¬ 
tions provided to reviewers were broad and general, and they were not changed 
much in these 1982 instructions: 

Without intending to restrict the scope of the review, we should like 
to have drawn to our attention what, in the opinion of the members 
of the Unit Review Committee, are the strengths and weaknesses of 
the department’s teaching and research programs, an assessment of 
the staff and staff workloads, the adequacy of resources provided, 
the effectiveness of the organization of the department, and the rela¬ 
tions of the department with others. In particular, we should appre¬ 
ciate having drawn to our attention any opportunities the URC 
perceives for improving the department’s programs, its relations with 
both internal and external organizations, and for better utilization of 
resources available. We also ask for your judgment of the department’s 
proposed plans for the future. 

The Unit Review Committee has a critical role. Its members must be wise and 
responsible as well as credible. Involvement of the department in nominating 
potential members carries the danger of collusion - that the reviewers will state 
that the quality of the department is better than it is, that deficiencies in support 
are worse. This danger is in reality minimal. The reviewers meet an exceedingly 
high standard if they are carefully selected. A team with such qualifications has 
a great deal of collective judgment to be tapped. Both general wisdom and scien¬ 
tific knowledge are desirable characteristics for all reviewers. Each team member 
brings a special viewpoint to the deliberations. The review team should provide a 
strong message and maximum impact. This objective is more likely to be attained 
if members are given freedom to investigate and report on what they find. 

Current status. Members of PACCR were selected in June 1980. In September 
1980 the review process started with a set of five diverse academic departments, 
which served well in the development of procedures. In the second year 15 more 
reviews were started. As of 31 March 1983, 51 units were in various stages: 12 
had been completed, 8 had completed the site visits, 8 had completed their self- 
study reports, and 23 were in the self-study phase. 

The process for reviewing any single unit has taken longer than anticipated 
particularly during the self-study phase. Nevertheless, completions are now to be 
expected at the rate of about 15 per year. For the future, continuing involvement 
with all major parts of the university is expected until completion. The present 
focus is to remain primarily with reviews of departments and smaller faculties. 
Later it may be appropriate to look at the larger faculties as organizations com¬ 
prising departments that have been reviewed. 

ISSUES IDENTIFIED IN REVIEWS 

During the review process at The University of Alberta from 1980-83, several 
issues have been identified. The major issues discussed below - time management, 
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reallocation of resources, teaching load, the teaching-research balance, and 
reviews of service/administrative units are referred to in the literature cited above 
or in articles dealing with university administration. 

The Time Problem 

Review teams are brought to the campus for a limited time, usually two days but 
sometimes three or four. About one third of the time is spent in the department, 
one third outside the department, and one third in committee discussion. Depart¬ 
ments normally want as much review-team time on site as can be allowed. The 
problem of limited time is expected to be a subject of continuing comment, 
particularly by academic departments. 

The time scheduled for on-site activity depends on several considerations. 
The larger and more complex units obviously require more time. Differences 
among faculties may warrant different review time on campus. The quality of 
the self-study report can also affect the time needed for adequate reviewing. 

Possibly not so obvious in the matter of on-site time are the conflicting 
interests of the department and the reviewers. Most of those nominated are highly 
respected, mature scholars who have many demandsontheirtime.Ourexperience 
has been that for two-day reviews external reviewers remain for the two full 
days. Assembling a team for three-or four-day reviews can be more difficult. 

Reallocation of Resources 

In common with other universities, The University of Alberta has suffered de¬ 
creased real support for.its operations, leading to increased teaching and research 
workloads. These concerns are obvious from statements made in the self-study 
reports. Claims from the unit are understandably often supported by the review 
team. But how can university administrators balance claims from “competing” 
units? What workload is fair and what is equitable? The University of Alberta 
has reserved a sum of money to allow the President to react positively to matters 
identified by the reviews. Even though the main purpose of the reviews is program 
improvement, and many types of improvement can occur without resource re¬ 
allocation, it is recognized that legitimate cases exist where improvement cannot 
occur without additional resources (Craven, 1980, pp. 451-452). Further, in view 
of the complexity of operations of the university, the President and Vice-Presidents 
are unable alone to assess the comparative validity of claims for additional re¬ 
sources and therefore understandably are turning more and more to members of 
PACCR for advice. This function, which goes beyond organization of the reviews, 
was not anticipated in the planning stages. 

The Teaching-Load Problem 

Most of the self-study and Unit Review Committee reports present points of view 
on the teaching load of unit staff. Even though numerical measures of teaching 
load are used in most universities, they need to be supplemented by informed 
judgment that comes to grips with these questions. Does load increase directly 
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with increase in number of hours of lectures? How does load vary as size of class 
varies? How does load in conducting seminars and laboratories compare with that 
in lecturing? How does load vary with course level — junior undergraduate, senior 
undergraduate, and graduate? Is some combination of hours of class contact and 
size of class desirable and if so how can an appropriate measure be derived? 

At The University of Alberta, as in other universities, a broad distribution of 
teaching loads occurs across departments. These differences arise or persist 
because of factors such as (a) the nature of the discipline, (b) the inability of the 
university to respond quickly to shifting loads with time, (c) differences in the 
aggressiveness with which chairmen have historically pursued resources, and (d) 
the impact of external funding for research, postdoctoral fellowships, and 
graduate students. 

The Research-Teaching Problem 

No sharp dividing line can or should be drawn between the two primary functions 
of teaching and research. In the broad context, when one is considering work¬ 
load, the separation of teaching and research may be introducing an inappropriate 
artificiality. As those instructed become more and more competent in their sub¬ 
ject, the activity grades imperceptibly, at least in many disciplines, from formal 
teaching through informal instruction to pure research. Thus, while teaching 
includes much more, in a narrow context it can be defined to include only the 
activities associated with the conduct of regularly scheduled classes. Research 
then includes the nonscheduled teaching of graduate students and postdoctoral 
fellows as well as other activities normally called research. 

We have found, however, that both the self-study and Unit Review Committee 
reports tend to consider teaching and research as separable functions. Such 
separation has also been sensed by Gonyea (1980, p. 369). In defence of this 
position, we recognize that when undergraduate teaching duties become excessive, 
the energy and time required to conduct research is no longer available. Similarly, 
the extent of commitment to research may preclude much effort devoted to 
the teaching function. 

From personal experience we are aware of some individuals who carry heavy 
teaching loads and are among the most productive in research. A reduction in 
teaching load does not necessarily improve research productivity. For most univer¬ 
sity staff, at least three months are available between winter sessions for research 
unimpeded by undergraduate teaching. Nevertheless, an important question 
remains, “How heavy must a teaching load be before it starts to interfere with 
research productivity?” 

Service/Administrative Units 

As only two service/administrative units have been reviewed to completion, it is 
difficult for us to generalize about reviews of this type. Reviews of these units, 
such as Purchasing, Personnel, Registrar, Library, Security, Student Services, 
Computing Services, and Housing and Food Services present problems different 
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from those experienced with academic units. In particular, because they usually 
have interaction with nearly every academic unit on campus, a larger number of 
groups are interested in providing input to the review team than with academic 
reviews. These units frequently deal with a more diverse range of activities than 
do many academic units, and this diversity can vary considerably across different 
universities. Not surprisingly, universities and colleges have typically concentrated 
more upon reviews of academic units. The attention given to reviews of service/ 
administrative units by The University of California and The University of 
Alberta is far from being common. Our experience, however, is that reviews of 
the support and administrative units are necessary and valuable. Our faith in the 
advantages of one committee organizing both types of reviews remains strong. 


CONCLUDING COMMENTS AND RECOMMENDATIONS 

For reasons related to accountability and self-interest, we need to examine criti¬ 
cally the operations of our postsecondary institutions. Consequently, organiza¬ 
tion of reviews of their programs will probably become a more common practice. 
These reviews need to be conducted with care and sensitivity so that long-term 
damage does not occur and maximum benefits are obtained. This need can be 
partially met by adoption of an appropriate, continuing review mechanism rather 
than ad hoc procedures initiated when trouble is perceived. 

The procedures adopted at The University of Alberta provide a “pure” example 
of the executive (presidential) model for conducting reviews. For three main 
reasons its approach is worthy of detailed consideration by other universities and 
colleges contemplating adoption of a system of continuing reviews: (1) the pro¬ 
cedures were carefully developed over approximately 30 months and benefited 
from the experience of many other institutions, (2) the procedures are simple, 
and the administrative/manpower costs are minimal, and (3) the procedures have 
required little alteration over three years as they have been well received by the 
departments which have been reviewed. 

Based upon expert opinion and the experience of The University of Alberta 
and. other universities, the following recommendations are proposed as some 
guidelines for universities or colleges contemplating program reviews as a means 
of helping to improve programs. These should be considered in association with 
the list of questions proposed by Craven (1980: 444-449). 

Control. The committee organizing the overall process should be selected by the 
President, be small in number (five or fewer), and consist of knowledgeable, 
respected and sensitive staff members representing both faculty and support/ 
administrative units. It will organize reviews of both the academic and support/ 
administrative units of the institution. It should keep in close contact with the 
department/unit at all stages of the review process. 

Purpose. While program improvement is usually the main purpose of reviews 
conducted by an institution, we should recognize that multiple benefits can result 
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from reviews and that program improvement may require reallocation of resources. 
By placing genuine emphasis upon improvement, a common initial perception 
that the main purpose of reviews is cutting programs, courses and staff can be 
corrected. Program review should not be used to deal with routine administrative 
matters such as poor leadership or poor teaching. These matters should be handled 
by invoking existing procedures. 

Procedures. The review procedures must be acceptable to the university/college 
community. This usually means that they are approved by the governing bodies 
of the institution. Procedures should be flexible, easily understood, and have 
minimum prescription. Although not all universities follow this approach, separa¬ 
tion of the committee organizing the reviews from the unit review committees 
is desirable, i.e., they should not have common members. The unit review com¬ 
mittee should be composed of appointed internal and external experts who 
possess sound judgment. 

All units on campus should be reviewed on a rolling cycle over a period of the 
order of a decade. A small number of units should be reviewed in the first year, 
the review process should commence with a self-study activity conducted by the 
unit. Standardized relevant data should be made available from central sources; 
these can be supplemented by other data specific to the unit’s operations. The 
self-study phase should take approximately six months and result in a detailed 
report of history, current operations, and plans for the future. 

Information should also be collected by the unit review committee from 
affected groups such as students, recent graduates, employers of graduates, and 
staff members in other departments/units. 

All reports resulting from the reviews — the self-study report, the unit review 
committee’s report, and the unit’s reaction to the review report — need to be dis¬ 
tributed to a restricted audience. Sensationalism is to be avoided. Senior adminis¬ 
trators should be responsible for ensuring that approved recommendations are 
implemented and for monitoring progress of such implementation. 

Our conviction is that a continuous system of program reviews involving academic 
and support/administrative units is essential and beneficial, that such reviews can 
help postsecondary institutions meet their financial challenges, and that the 
information from these reviews is a necessary adjunct to sound university planning. 
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